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The measured forward-backward asymmetry in the tt production at the Tevatron might be ex- 
plained by the additional exchange of a colored weak singlet scalar. Such state appears in some of 
the grand unified theories and its interactions with the up-quarks are purely antisymmetric in flavor 
space. We systematically investigate the resulting impact on charm and top quark physics. The 
constraints on the relevant Yukawa couplings come from the experimentally measured observables 
(<— s related to D^-D° oscillations, as well as di-jet and single top production measurements at the Teva- 

I tron. After fully constraining the relevant Yukawa couplings, we predict possible signatures of this 

f^ model in rare top quark decays. In a class of grand unified models we demonstrate how the obtained 

^sj information enables to constrain the Yukawa couplings of the up-quarks at very high energy scale. 

K*" PACS numbers: 14.65.Ha,12.10.-g,14.40.Lb 
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^~' I. INTRODUCTION 

i-C During the last decade, rare processes of down-type quarks have been proven instrumental in the study of physics 

'"' beyond the Standard Model (SM). On the other hand, the discovery of novel effects in the up-quark sector is still 

O I often dismissed as very unlikely. In the SM in particular, rare charm processes are mostly dominated by long distance 

D dynamics, the short distance contributions being subject to severe Glashow-Iliopoulos-Maiani cancellations. This is 

r^ ^ the case in D^-D^ oscillations as well as in the c^ wy and c -^ ul^l^ decays. Nonetheless, the experimental situation 

in the charm sector is already very restrictive and present measurements constrain many models of new physics [IH7] . 

(T^ The heaviest up-type quark, i.e., the top quark, has been carefully investigated since its discovery in 1995. Although 

^ enormous progress has been achieved, the prospects for observing flavor changing neutral current mediated top quark 

'^ decays at the Large Hadron Collider (LHC) or Tevatron are rather weak. The LHC experiments might be sensitive to 

^^ the branching fractions of the order of 10~^ or less. However, the majority of well studied models of new physics (NP) 

«^ predict much weaker signals. It thus seems that the search for NP in the top sector should be directed towards other 

processes [3 [9]. 
^^ The recent CDF measurement of a large positive forward-backward asymmetry (FB A) in the tt production disagrees 

with the SM prediction and thus indicates a possible presence of NP. Among the many proposed NP scenarios 
aiming to explain this discrepancy, we have recently suggested [lOj a theoretically well-motivated SU{5) grand unified 
theory (GUT) jTT] model [12]. The model has an appealing feature of correlating the presence of light colored scalars 
stemming from a 45-dimensional Higgs representation with bounds on the proton lifetime. Namely, the aforementioned 
representation contains among other states two colored scalars — Ag = (3,1,4/3) and Ai = (8,2,1/2) — that have 
jj] masses below or of the order of 1 TeV when partial proton decay lifetimes are predicted to be at, or slightly above, the 

Cu current experimental bounds. In this regime Ag can help to reconcile the SM theoretical prediction of the forward- 
backward asymmetry in ti production [101 [131 HI , which is more than 2a below the measured value, while preserving 
the agreement in the total tt production cross section. On the other hand, the contributions from Ai are required to 
be suppressed }10j . 

Relatively light colored scalars may also also arise in other extensions of the SM. Generically, lower bounds of their 
masses can be inferred from their strong interaction mediated production at hadron colliders. In addition however, 
their couplings to SM matter fields will induce contributions to flavor observables. We investigate constraints and 
predictions of flavor observables in the up-quark sector in the presence of a color triplet weak singlet scalar with 
hypcrcharge 4/3. Furthermore, we address the resulting constraints on the Yukawa sector of a particular class of 
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GUT models that employ the 45-dimensional SU{5) Higgs representation containing such state [TS]. As has been 
noticed recently, Ag exchange does not contribute to d = 6 proton decay operators p,6j and can thus be light. 



In Section [nj we describe our framework. The D^-D^ constraints are studied in Section III Constraints coming 
from the Tcvatron single top and di-jet production cross-section measurements are presented in Section IV Resulting 
predictions for rare top decays are given in Section Ivj while implications of phenomenologically deduced constraints 
for the entire up-quark Yukawa sector in GUT models are addressed in Section |VI[ Discussion and summary are 
presented in Section [Vll[ 

II. FRAMEWORK 

The color triplet scalar we study appears in theoretically well-motivated class of SU{5) models. Namely, it is a part 
of a 45-dimensional Higgs representation, which has been frequently used in GUT model building to accommodate 
known fermion masses, improve unification of gauge couplings, and address proton decay constraints |12l 1151 IT7U20) . 
Recall that matter of the SM is assigned to a 10- and 5-dimensional SU{5) representations, i.e. 10^ = (1,1,1) © 
(3, 1, —2/3) ® (3, 2, 1/6) and 5^ = (1, 2, —1/2) ® (3, 1, 1/3) where i = 1, 2, 3 denotes generation index. It couples to the 
45-dimensional Higgs representation 45 through the following operators 

^matter ^ ^YifJ ^io^Py^^5,)^45*J^ + (ra)'^'^^/?^^. (10"^),(10^''),45^^^ (1) 

Lepton and baryon number violating Yukawa couplings of Ag in the interaction basis are 

£a6 = V2[{Y2)^J - {Y2)j^]eabcU^aPLu'jbA'e + {Y^Y^ e^PLd^A^* -t- H.C., (2) 

where a, 6, c are color indices, while J, j denote flavors. Note that the antisymmetric nature of the Ag coupling to 
the up-quark sector in flavor space is dictated by group theory. Yukawa couplings of Ag to diquarks in the physical 
basis, i.e., the up-quark mass eigenstate basis, acquire a unitary rotation (Ur), which however does not spoil the 
antisymmetry in flavor indices. Hence, we deflne 

gl^ ^ 2^2 [ul{Y2 - y/)C/;,] '' , gl^ = -gt (3) 

altogether with three independent parameters: g}^ , g^^, and g^'^. It is the antisymmetry of gg that is responsible for 
the absence of the d = 6 operators due to Ag exchange that would otherwise contribute to proton decay |16j . In what 
follows we focus our attention on phenomenological constraints of these entries. 

Note that a tree-level exchange of Ag contributes to ii production cross-section in the u-channel. It was emphasized 
in 'W that Ag at and below 1 TeV can enhance the SM prediction of the forward-backward asymmetry Ap^ while not 
altering the production cross-section cr^f . This has been achieved by flnding the parameter space of m^^ and coupling 
gl'^, contributing to partonic subprocess uu — > ti, in pp collisions, which together with SM reproduce the measured 
values of A"^ and Ctt . The region where experimental constraints can be satisfled within la roughly corresponds to 
a region, where mass of Ag and the coupling ^g'^ are correlated as 

|5g"|= 0.9(2) + 2.5(4)^. (4) 

We investigate the bounds on the remaining two couphngs {g^^, gl^) in the following sections. 

III. D°-D" MIXING CONSTRAINTS 

Mixing of neutral charm mesons is sensitive to interactions which change charm flavor by 2 units. Scalar Ag, if light 
enough, may strongly contribute to effective |AC| = 2 interactions and expose its Yukawa couplings to the up-type 
quarks in D^-D^ mixing. In this section we study the contribution of Ag to D^~D^ mixing observables and place 
constraints on its Yukawa coupling to c and t quarks — g1^ . 

The D^-D^ mixing matrix element of |AC| = 2 Hamiltonian is split into dispersive {M12) and absorptive (ri2) 
parts as following 

(Z?"|H|5") = A/i2-^ri2, (5) 



and due to the (anti)hermiticity of My (i/2 Fy) also (Z)° I "H | D^') — MI2 — fr*2 holds. Two dimensionless parameters 
describe magnitudes of the dispersive (a;i2) and the absorptive {yi2) part while a relative phase between AI12 and ri2 
would signal CP violation [3] 
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Here T is the average decay width of neutral D mesons. Contributions to the absorptive part j/12 come from on-shell 
intermediate states and are thus accounted for by SM. On the other hand, in the presence of Ag a;i2 and (j)i2 are 
affected by the box diagrams consisting of Ag and i-quark exchanges which mediate D^-D^ transitions, as shown on 
Fig. [1] Since the preferred mass of Ag is between 300 GeV and 1 TeV in order to explain the ti asymmetry [TD] , we 





Figure 1. Contribution of order (gl'^ge^*)^ to AC — 2 effective Hamiltonian. 



can safely integrate out both the top quark and Ag at a common scale /i ~ rn/^^ . Leading order matching onto the 
effective theory generates a single operator, denoted in the literature as Qg [2T| 



1-L{ji = niAe) = Ce{mAe)Q6, Q& = {uii'-f^'cB,){uRjf^CR), 

with the corresponding Wilson coefficient 
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Effective Hamiltonian is evolved down to the charm scale fin = 2 GeV using the leading-log anomalous dimension 
The multiplicative renormalization factor of Cg in our case is adopted from Ref. [1 and reads 
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The nonperturbative bag parameter Bu corrects the vacuum insertion approximation value of the mixing matrix 
element 



{D°\{uRrcR) iuRjf,CR)\D°) = -mlflBD. 



(10) 



We use the value Bni^J^D — 2 GeV) — 0.785 calculated on the lattice employing quenched Wilson fermions [22]. For 
the decay constant we use the CLEO measured value fo — 0.206 GeV [53]. 

Experimentally, D^-D^ mixing has now been confirmed, while GP violation is still consistent with zero, as is 
evident from the latest HFAG [M] average which assumes no direct CP violation 



x = (0.59 ±0.20)%, 
\q/p\=i)Mt°,\l 



y= (0.81 ±0.13)%, 
= -0.05liril'5 ■ 



(11a) 
(lib) 



The assumption of negligible direct CP violation resides on the fact that Ag cannot contribute to F12, and that in the 
SM CP violation is very small [3 . In this case, the four measured parameters are governed by the three independent 
theoretical quantities Q, implying redundancy of one of the four experimental parameters. Indeed the values of \q/p\ 
and 4) are found to be strongly correlated in the HFAG fit (11) with correlation coefficient of 0.614 [21]. We choose 



to extract a;i2 and 4>i2 from x, y, and \q/p\ using the following relations [31125] 

2 _{\q/p\' + lYx' + {l-\q/pn'y' 
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Imaginary part of AI12 is directly accessible in the product 



a;i2Sin0i2 



2 ImMi2 



(14) 



which we constrain, along with X12, from the HFAG values (11 1. The 1 and 2cr confidence levels of X12 and 2:12 1 sin(/)i2| 
are shown on Fig. l2J whereas the upper bounds at 95 % (2(t) confidence level are 
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Figure 2. The 1 and 2a confidence level regions (dark and light-shaded regions, respectively) in the plane a;i2-a;i2| sint 



a;i2 < 9.6 x 10"^ 2:12! sin </)i2| < 4.4 x 10"^. 

The imaginary part of M12 originates from relative phase between the g^^ and g^^, namely, 

lm[{gl'grr]-\9l'\'\9r\'M2u.), 



(15) 



(16) 



where ui is the difference between phases of ^l"^ and g|^. Using the central value for \gl^\, as given in Eq. Q, we find 
the region of l^l'^l, which is limited from above by the 95% upper bounds (15), and is shown in Fig. p] The bound 
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Figure 3. Allowed regions of \g'i'^\ and lj for different values of m^g. 

on X12I sin 012 1 is stronger, except in the regions close to w = or 7r/2, where bound on 0:12 dominates. Apparent 
nondecoupling, i.e., strong bound even for large masses m-Aei is due to linear dependence of \gl'^\ on mAg (see Eq. ^). 
We obtain a robust 2cr bound in the region of interest m^g < 1 TeV: 



Igfl < 0.0038, 



(17) 



regardless of complex phases, thus allowing also for finely tuned phase w. 
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Figure 4. Partonic contribution to di-jet production at the Tevatron from a u-channel Ae exchange. The analogous s- and 
f-channel contributions can simply be obtained via crossing. 



One can extract the product \ge^gl^*\ also from the radiative c ^ uj decay, where the Ag-induced effective 
Hamiltonian reads 



n ^^ ^(4^("^^ ^^)^^- 

with F^^, e the electromagnetic tensor and coupling, whereas function / is 



,f(^) 



2x^ + 3a;^ — 6x^ log a; — 6x + 1 



(x-l)4 
However, using bounds on the couplings (Eqns. ( |17p and Q) we find 



(18) 
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whereas the SM resonant contributions to Z? — >■ V^ and D -^ P£^i^ {P and V denote pseudoscalar and vector 
mesons) are of the order of 10~^ [2SH2H]- This implies that radiative decays offer a far weaker constraint than the 
measured D^-D^ observables. 



IV. BOUNDS ON gl^ 

In this section we consider bounds on the coupling g^^ coming from the CDF search for resonances in the invariant 
mass spectrum of di-jets [29] as well as from the single top production cross-section measurements at the Tevatron [30j . 
The first measurement constrains the \g]^\ coupling directly since the process can be mediated by Ag through s-, u- 
and i-channel exchange diagrams (see Fig. [4]) interfering with leading order (LO) QCD contributions at the partonic 
level. The resulting partonic uu — >■ cc differential cross-section from the w-channel Ag contribution interfering with 
the s-channel single gluon exchange is 
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(21) 



where s = {pu +Pu)'^i t — (p„ —PcY', u — [p^ ~ PcY ■ The expression for the cc — ^ uu is identical, while contributions 
from the related uc — > uc, cu — )■ cu, uc — >■ uc and uc — ^ uc processes can be obtained via crossing. Since the last two 
contributions involve Ag exchange in the s-channel, we have included the Ag total width Fab as the regulator of the 
on-shell pole in all expressions. We assume that Fab is dominated by the three decay channels to pairs of quarks ut, 
ct and uc. For the Ag — >■ tqi {qi,q2 = u,c) channels we keep the top quark mass dependence in the partial width 



F(Ag ^ %) 



1.96 



i3\2 
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(22) 



while we neglect quark masses in the analogous expression for the Ag — )• uc channel. As shown in the previous section, 
the tc channel is severely constrained by experimental results on D^-D^ oscillations and can be neglected. In order to 
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Figure 5. Partonic contribution to tlie single top production at the Tevatron from a ii-channel Ae exchange. The analogous 
s-channel contribution can simply be obtained by crossing. 

compare with the experimental results |29| we first compute the hadronic di-jet production invariant mass spectrum by 
convoluting the LO QCD partonic differential cross-section including the above described tree-level Ag contributions 
with the CTEQ5 ^Tj set of parton distribution functions (PDFs) at the factorization and as renormalization scale set 
to i-iF = ^iR = Et, where Et is the transverse energy of the final state di-jet. (We employ as{mz) = 0.117 J32l.) We 
also require both jets to have rapidity \y\ < 1. Then we reweigh our results so that our SM predictions (with gl"^ = 0) 
match the complete NLO QCD results computed with FastNLO ^33", CTEQ6.1 |3T set of PDFs and appropriate jet 
algorithms as used in the original CDF analysis |29) . Finally, we turn on Ag contributions and compare the resulting 
spectra with published CDF results. Assuming that Ag is responsible for the measured large FBA in top quark pair 
production at the Tevatron |55H57] , combined with the measured total tt production cross-section as well as the top 
pair invariant mass spectrum, leads to the g^^ coupling and Ag mass satisfying the approximate relation given in 
Eq. Q. Then the total Ag decay width in the interesting region of |gg^| is comparable with the experimental di-jet 
invariant mass bin size. We obtain the bounds on |gg^| as a function of rriAs by comparing the obtained theoretical 
spectrum for given values of Ag parameters with the experimental measurement. The results are shown as the purple 
shaded area in Fig. [6] 

The single top production cross-section is sensitive to the product of |gg^5g^*| (and also \gl'^ge^*\, which, however, 
is more severely constrained by D^-D^ oscillation measurements and we neglect it in what follows) since it proceeds 
at partonic level through s- and u-channel Ag exchange diagrams (see Fig. pi). The resulting w-channel partonic 
contribution to uu — >■ tc has the form 

where now s = {pu +Pu)^, i = (Pu — PtY, u = {pu — Pt)'^ ■ The analogous uc — > tu expression is related to it via 
crossing. A bound on \g\'^\ can then be obtained by using the tt FBA preferred values of \g\^\ in Eq. (Wl). Since all 
the current experimental measurements of single top production use sophisticated multivariate analysis techniques it 
is difficult to compare Ag contributions directly with their data. Therefore we employ a conservative approach and 
only compare NP contributions with the experimental error on the combined Tevatron result for the total single-top 
production cross-section (summed over both t and i) of an — 2.761q;|7 P^ [SQ] — we require the Ag mediated hadronic 
production cross-section, which we denote Acru, to be smaller than 1 pb at 95% confidence level. A recent dedicated 
DO analysis of NP mediated single top production obtained even stricter limits on the anomalous single top production 
cross-section [38] . however their results are not directly applicable to our case due to different partonic initial and 
final states as well as different kinematics. We compute the NP-mediated single top production by again convoluting 
the partonic differential cross-section with the appropriate PDFs at the factorization and a^ renormalization scale 
set to /ii? — ^n — Et and then integrating over the available hadronic phase-space. Since in this case there are no 
contributions due to the SM-NP interference, the Ag mediated single top production cross-section (and the associated 
constraint) scales quadratically with |5g^(?g'^*|. Using the tt forward backward asymmetry preferred values of \gl'^\ in 
Eq. Q we obtain the bounds on \gl'^\ shaded in blue in Fig. p] 

V. PREDICTIONS FOR RARE TOP QUARK DECAYS 

Having obtained the upper bounds on j^g^l and |g|^| we can now assess the prospects of observing radiative top 
quark decays at the LHC. We predict decay widths of processes with an up- type quark and a photon or a gluon in 
the final state. 
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Figure 6. Constraint on the Igg^j coupling and Aq mass from the single top production and di-jet at the Tevatron. The shaded 
areas are excluded. 



The Ag-induced effective Hamiltonian for a top quark decay to a light up-type quark (massless c or u) and a single 
photon or a gluon reads 



1jt^qn,qiG 12 -^ ("^Afi/™*) emt r 13*/ p^, x „23*/- „tiu, \1 p 
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[5^'*(cKfTM-r^^i) -56''*(^fl^M-r^^L)] G^, 



where e, ^F^^, and g, Gf;,^ are electromagnetic and color coupling constant and field tensors, respectively. Dependence 
on TTiAs is ascribed to 
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(25a) 
(25b) 



The light quark in the final state, regardless of being either c or u, couples via g\'^ to the light quark and Ag in the 
loop. The top quark, however, couples predominantly to u and Ag (its coupling to c and Ag is suppressed by the 
smallness of l^g^l) in the loop and thus t -^ uj, uG decay widths are suppressed by a factor of Iffi'^/ffe^P ^ IQ^^ with 
respect to t — >■ C7, cG widths. For decays with a c-quark in the final state we find at leading order in as 
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(26) 



For Ag masses below 1 TeV the expressions (26 1 lead to branching fractions' upper limit of the order of 10~^ for both 
t ^ C7 and t — > cG. Independently of the coupling constant values, the width of the photonic channel is 30-40 % 
larger than the t — > cG width. QCD corrections for these processes are known ^35"-41J. For the t — ?> cG channel they 
amount to 20 % enhancement in the branching fraction. On the other hand, the smallness of the gluonic rate leads 
to negligible effects in the i — >■ C7 branching fraction [1D| . 



Interestingly, the ratio of tree-level contributions of Ag in single t production cross-section Acrit (see Eq.(23)) and 
t — )■ C7 or t — ^ cG decay width is completely independent of gg couplings. On the other hand, the ratio Tf^^Q^^^/ /S.ait 
exhibits a nontrivial Ag mass dependence, as shown in Fig. [7] 



VI. PREDICTION FOR THE UP-QUARK YUKAWA SECTOR 



The fact that all three entries of gg are either fixed or bounded through processes that involve the same scalar field, 
i.e., Ag, presents us with a unique opportunity to put constraints on relevant Yukawa coupling constants within a 
large class of GUT models in the up-quark sector at the very high scale as we show next. 
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Figure 7. Ratio of top quark decay widths t — > 07, cG and Ae contribution to the single top production at the Tevatron. See 
text for explanation. 



We mentioned already that Ag is a part of a 45-diniensional Higgs representation of SU{5) and the 45-diniensional 
Higgs representation, i.e., 45, is frequently used in SU{5) GUT models. The relevant coupling — already given in 
Eq. (fTl) — basically sets a model dependent lower bound on a vacuum expectation value of the Higgs doublet that 
resides in 45. 

We find that at least one component of ge that describes the couplings of Ag to the up-quarks must differ from 
zero to accommodate forward- backward asymmetry in top quark pair production at the Fcrmilab Tevatron. But, ge 
is related, via Eq. ([3|, to the coupling of the Higgs doublet in the 45-dimensional representation to matter. This 
antisymmetric contribution, however, is not sufficient to generate viable up-quark mass spectrum. Namely, one 5- 
dimensional Higgs, i.e., 5, is also needed to generate viable up-quark mass spectrum via renormalizable set of SU{5) 
Yukawa coupling contractions. The relevant contraction of 5 with matter is eQ^-y5e(y2)jj(lf"^)i(10^*)j(5)'^. The 
up-quark mass matrix at the GUT scale reads 



Mu = [4(1-2'^ + Y^)v, - S{Y^ - Y^)vi^] /V2, 



(27) 



where 

|2 



|2 _ 



and 



(45)fi 
'^ where v 



(45)2^ = (45) l'^ — a/2^'45 represent appropriate vacuum expectation values. (Note, 
2|w5|^ + 48|w45|'^ = v^ ^ where v = 246 GeV.) Y2 and I2 are in general arbitrary 3x3 Yukawa matrices. 

We can now show that constraints on g^ immediately translate into constraints on symmetric coupling contribution 
in Eq. ( 27 ) . Namely, if we go to the basis where the down-quarks mass matrix is diagonal we obtain the following 



two equations 
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A A' = uHlfi'"'' - Mf'^'^U*, 



(28a) 
(28b) 



where U — VckmUr and M^""^ is a diagonal up-quark mass matrix. Note that Vckm = UiVckmU2 is proportional to 
Cabibbo-Kobayashi-Maskawa (CKM) matrix Vckm apart from five phases that are present at the GUT scale contained 
within diagonal unitary matrices C/i and U2. A'{= 2V2UJ^{Y2 - Y^)U;^V45) and S"(= V2UJ^{Y^'^ + Y^)U'^V5) are 
antisymmetric and symmetric contributions, respectively. Ur, again, is an arbitrary unitary matrix. These relations 
hold at the GUT scale whereas our constraints on ge originate from the low-energy phenomenology. If we run ge 
constraints as well as relevant fermion masses and their mixing parameters from electroweak scale to the GUT scale we 
would finally have A' — ggVis- It is this relation that lets us deduce generic properties of the symmetric contribution 
S' and thus pinpoint the texture of the up-quark Yukawa couplings. 

We now state several important generic features of the Yukawa coupling constants in the up-quark sector as implied 
by our constraints on gg- Firstly, the ratio l-S^gl/jA'^gl is equal to one at all instances. This is due to the fact that 
one of the \A'\ entries, i.e., the one proportional to |(7g^| is fixed phenomenologically and is significantly larger than 
the upper bounds on the other two entries of \A'\. Further, both \A'i^\ and \S'i^\ are well approximated with the same 
product |C/3i|7TT.t since M^""^ is highly hierarchical matrix. Since |(7g^| is fixed and larger than both j^l^l and {g^^ 
find IC/31I/V45 to be just a constant for a given m^g while \S[^\ = \A[r^\. 



we 



Secondly, since 1^23 1 should phenomenologically be much smaller when compared with \A[^\ it is clear that |C/; 



32 I 



entry must be very small in order to suppress the first term in A23 — 1/32"^* ~ ^23 "^c equality. As a consequence, there is 
thus a correlation between IC/31I and If/33 1 due to unitarity of U. This correlation also implies that the ratio IS'ggl/jA'j^gl 



9 

should depend only on \U3i\ (or, interchangeably, 1145). In fact, analytically we find IS'ggl/IA'igl « 2a/1 — IC/31P/IC/31I. 
We plot IC/31I vs. |S'33|/|A'j3| as obtained in our numerical analysis superimposed over 2a/1 — IC/31P/IC/31I in Fig. ^ 
Squares represent satisfactory numerical solutions of Eqns. (28) that satisfy all other phenomenological constraints. 
We describe our numerical procedure in detail later on. The most important facts we can conclude from Fig. [8] are 
that IC/32I is indeed negligible, 15*331 i^ determined for a given value of IC/31I and 1 5*331 is usually smaller than or, at 
most, comparable with {A'^^l. 

Thirdly, there is a connection between I5J2I = |t^2i'^c + Ui2'nT'u\ and \Ai2\ = \U21mc — C/i2'^m|- Namely, if IC/31I 
is small then unitarity makes IC/21I large which in turn implies that the ratio 15^21/1^121 tends to one. In the other 
extreme, when |[/3i| approaches one, IC/21I goes to zero and there is a possible interference between the two terms in 
15^2! and \A'i2\ which makes their ratio deviate from one but not by a large amount. This behavior is depicted in 
Fig. [olwhere we plot log 15*^21/1^121 ^^- l^^ail- Clearly, 15^21 and \A[2\ are within a factor of three from each other for 
all values of |f73i|. Note, IC/31I cannot be arbitrarily small as is also evident in Figs. Is] and l9] In fact, when IC/31I —?> 0, 
with IC/32I negligible, \A[^\ -^ 0. 

Fourthly, there exist a correlation that concerns \S[j^\ and |522|. What we find is that, due to the fact that IC/32I is 



negligible, unitarity of U demands that |5ii|/|522| -^ rriu/mc for IC/31I — >■ 0. We plot this behavior in Fig. 10 where 



we plot log |5*ii 1/15*22 1 vs. \U3i\. It is clear from Fig. 10 that IC/31I 7^ 1 yields unique value for |S*(i|/|522|. Recall, in 
that regime, the ratio |5^2 1/1^12 1 is also uniquely determined. 

Finally, there exists a model dependent lower bound (and upper bound) on 1745 from the charged lepton and down- 
quark sector in models with the 45-diniensional Higgs. However, due to the fact that the 45 contraction on its own 
cannot provide viable up-quark masses it is clear that there will also exist an upper bound on W45 from the observed 
up-quark sector masses. The novelty is that this upper bound will be iriAe dependent since the phenomenologically 
viable form of ge is toab dependent as given in Eq. (HI). We show the upper bound on W45 as a function of mAg in 



Fig. [Ill As expected, it drops as ttiab grows in order to suppress corresponding growth of \gl^\. 

Before we turn to numerical analysis we stress that all our previous observations do not depend on exact values of 
up-quark masses. 

A. Numerical analysis 

To generate our numerical results shown in Figs. [8j [9j [T0| and [TT1 we have turned to a specific scenario comprising 
45 = (Ai,A2, A3, A4, As, As, At) = (8,2,1/2)0(6,1, -1/3)0(3, 3,-1/3)0(3, 2, -7/6)0(3,1, -1/3)0(3,1, 4/3)® 
(1,2,1/2), 5 = (^c,*t) = (1,2,1/2)0(3,1,-1/3) and 24= (Eg, S3, £(3,2), ^(3,2), S24) = (8,1,0)0(1,3,0)0 
(3, 2,— 5/6)0(3, 2, 5/6)0(1, 1,0) of Higgs and one fermionic adjoint representation [12 24i? = {ps, Ps, P {3,2), P (3,2)1 P24) = 
(8, 1, 0) (1, 3, 0) (3, 2, -5/6) (3, 2, 5/6) (1, 1, 0). RecaU, Ag can be light in this model without supersymmetry 
that employs 24-dimensional fermionic representation [12] to generate neutrino masses via combination of type I [42l - 
135] and type HI [371 US] seesaw mechanisms. (This is a renormalizable version of the model first proposed in [35] and 
further analyzed in [SHI [SI]-) In fact, we have demonstrated [TD] that this scenario predicts proton decay signatures 
that are very close to the present experimental limits due to partial proton decay lifetime measurements when both 
Ai and Ag are in the range accessible in collider experiments. 

Numerical procedure we use to generate particle spectrum aims at an exact 1-loop level gauge coupling unification 
scenario with maximal possible unification scale Mqut- R allows for multiple particle thresholds as long as all relevant 
constraints are satisfied. In particular, we require that 10^ GeV< rriSg , ra^.^ , m^^ , toaj , rn^^ , ttiay , ^np^ , rUp,^ 3, , rup^ < 
Mgut, 10^^ GeV< m^y,mA3,TOA5 < Mqut and 10^ GeV< mpf^ < Mqut- Note, the masses of '^t, A3 and A5 are 
bounded from below due to the fact that these fields mediate proton decay through the d = 6 operators, pg, on the 
other hand, should be heavier than 10® GeV to accommodate the Big Bang Nucleosynthesis constraints [49]. Finally, 
the allowed couplings of the fermionic adjoint lead to the following mass relations [T2] 

il + rh) .. 

iTip, = mmp3, mp,3^, = irip^^^^^ = rUp^, (29) 

where to represents a free parameter that describes the mass splitting between pg and p^ states. 

We take the relevant parameter for the adjoint fermion mass spectrum to be to = 10^^ and set to-a^ = 400 GeV 
and toai = 1 TeV to generate particle spectrum of the theory. This set of assumptions together with a^ = 0.1176, 
a-^ = 127.906 and sin^ Ow = 0.23122 [32] yields Mqut = 1-1 x IQi® GeV and a^ljj. = 26.9, where aauT represents 
unified gauge coupling at Mqut- The corresponding particle spectrum reads toa, ~ 2.6 TcV, TOA3 — 10^^ GeV, 
TOA4 = TOA5 = 1.1 X lO^^GeV, TOA, = lO^GeV, ttit.., = 10^ GeV, toe, = 10^ GeV,"TO*^ = 1.1 x lO^® GeV, to^, = 
5.6x 10"^^ GeV, rUp^ = 113 GeV and TOp^g ^^ = rup = l.lx 10^® GeV. Using that spectrum we find the following values 
of the up-quark masses and CKM parameters at the GUT scale at 1-loop level: TOu = 0.00046 GeV, nric = 0.182 GeV, 



10 



rrit 



^CKM 



0.225, 



^CKM 



0.00394, sfg^*^ = 0.0462 and S^'^^' = 1.185. The input values at the Mz 
CKM _ n 0070 ^CKM _ Q Q^22 gC^^i 



55.4 GeV, s^^ 
are m„ = 0.0016 GeV, m^ = 0.628 GeV, m.t = 171.5 GeV, s'{^'^' = Q.2TI2, s'^^'^ = 0.0422, s'(^'^' = 0.00399 and 
gCKM _ Q 9g5_ ^11 relevant input parameters, i.e., quark and lepton masses and mixing parameters, and running 
procedures are specified in Ref. [TUl . 

In this particular regime proton decay contributions are dominated by the color triplets that reside in both 45 
and 5. (See Fig. [1] in Ref. [lO].) However, to be able to accurately predict partial decay lifetimes we need to know 
the coupling strengths of the triplets to the matter. Here, again, we demonstrate that one part — the one related to 
the up-quark sector — is already well known due to low-energy constraints. We have done the running of all relevant 
parameters from low scale to high scale only for the mAg = 400 GeV case and used those values to infer that the 
overall Yukawa coupling drop from low-scale to the GUT scale is well described by a common factor of 3.5. It is this 
factor that was used for analysis for different values of toab- We find this approximation well-justified since our main 
goal is to present generic features of the up-quark Yukawa sector at the GUT scale. 

We randomly vary angles and phases of U to see if relevant constraints on the elements of A' are satisfied as given 
in Eqns. Q, (15) and in Fig. [6] adjusted by a factor of 3.5. Once this is done we read off phenomenologically allowed 
elements of S". Our procedure guaranties viable values of the up-quark masses and CKM mixing parameters while A' 
components satisfy all low energy constraints. Note, since we know Vckm at the GUT scale, we can also reproduce 
Uu (up to some phases of diagonal unitary matrices). 

All in all, we have generated 10* random points in the nine-dimensional space of the unitary matrix U for different 
values of Ag mass, i.e., toab = 200, 400, 700, 1000 GeV. For example, when toab = 400 GeV we find that out of 10* sets 
of initial values of parameters of U, i.e., the angles and phases of unitary matrix, only 657 pass all phenomenological 
constraints which implies rather unique form of Yukawa couplings. Due to the fact that our antisymmetric matrix A' 
is highly skewed with \A'-^^\ element being dominant and the fact that the symmetric element 15*^31 is much larger than 



both 1 5*^2 1 and |S'23l elements we have a situation that the underlying matrix Mu as given in Eq. (27) in the down- 



quark mass eigenstate basis is of the so-called lopsided form. (Recall, 15*^31 or equivalently \A[^\ is either comparable 
or larger than IS'ss].) 

Our results for the underlying Yukawa structure of the up-quark sector exhibit dependence on W45 (or IC/31I = 
4:\gg'^\v45/mt). This, as we discuss, provides basis for analytic correlation between all entries of S" and A' , except for 
|523|. In order to appreciate the advantage of this analytic study, we bracket the range of values that these entries 
can take by resorting to our numerical analysis. We present these ranges, without correlation, for ttiab = 400 GeV 
and rriAg = 1 TeV in Table IT] for the absolute values of components of antisymmetric and symmetric contributions. 
Note that the ranges are more or less ttt-As independent since g^ elements scale similarly with respect to change in 
TOAe in tiiG region of interest. 



Table I. Allowed ranges of absolute values of 



relevant components of A' and S' in nit units for rriAe = 400 GeV and niAe = 1 TeV. 



A'lal/mt 
A'lsl/nit 
A2s\/mt 



S'lil/nit 
S'i2\/mt 

S'22\/mt 
S'23\/mt 

^33 I /^i 



ruAe = 400 GeV 
[2.9 X 10"^8.0 X 10"*] 
[3.7 X 10-^2.5 X 10-1] 
[1.4 X 10-^5.6 X lO""] 
[2.2 X 10-^4.0x lO"*^] 
[2.6 X 10~^8.1 X 10"*] 
[3.7 x 10-^2.5 X 10"*] 
[5.9 X 10-^1.7x 10"^] 



[5.3 X 10~''',3.9x 10"^] 
[6.5 X 10"^8.2 X 10""*] 
[1.8 X 10"^2.5 X IQ-*] 
[1.0 X 10"*, 1.6 X 10"^] 
[1.7 X 10"^ 2.2 X 10"^] [3.2 x 10"^ 2.1 x 10"'^] 
n 9 V in-3 /I Q V in-il r-? ■:; V in-3 <^ n ^ in-il 



mAe = 1 TeV 



[1.1 X 10"^8.2 X 10"*] 
[1.8 X 10"^2.5 X 10"*] 
[1.4 X 10'^5.1 X 10"*] 



X iU J [J./ X iU ,/.i X iU J 

[1.2 X 10"^4.9 X 10"*] [3.3 x 10"^5.0 x 10"*] 

To summarize, we have shown that the phenomenological constraints on the form of A' put limitations on allowed 
form of S", i.e., the symmetric contribution to the up-quark masses. In fact, \S[2\y \S[^\, {S'^^l and |S'ii|/|>5'22l are tied 
to the value of one parameter only, i.e, If/31 1 (or, equivalently, W45). We have shown corresponding ranges of values 
of \A'\ and |S"| elements in Table IT] for all possible V45. Clearly, V45 cannot be determined although its upper bound 
drops as a function of niAe- This bound will merge with the lower bound on W45 for a rather large value of niAg, 
when Ag is out of reach of accelerator experiments and |gg^| becomes too large to be trusted. However, additional 
constraints on the form of Yukawa couplings in the charged lepton and down-quark sector would further reduce the 
available parameter space for W45. If and when 1145 is better known, we would be able to bracket 5" elements within 
a narrower range of values. 

Note that Ag, being a part of 45-dimensional representation of SU{5), is also a part of a 120-dimensional represen- 
tation of 5'O(10) |521 153| that is frequently used to generate antisymmetric Yukawa contributions to charged fermion 
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Figure 8. Plot of \Uzi\ vs. IS'ssl/l^'is]. Squares represent results of our numerical analysis as described in the text whereas the 
curve stands for approximate analytic expression 2-\/l — lt^3iP/|t/3i|. The observed agreement implies that \U-a2\ is negligible. 
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Figure 9. Plot of log |S(2|/|^i2l vs. \U-i\\. There exist a lower bound on |[/3i] due to the fact that \A'yj,\ > IA23I. See text for 
details. 



masses. There, in the iS'O(lO) framework, the symmetry dictates that the antisymmetric contributions to charged 
leptons, down-quarks and up-quarks are all proportional to the one and the same underlying Yukawa coupling matrix 
that, on the other hand, is proportional to g^,. In such a setup one might have additional constraints that could 
prove sufficient enough to pinpoint the exact Yukawa structure. In fact, the lopsided structures within the 5*0(10) 
framework are known to connect small angles in the quark sector with large angles in the leptonic sector |54j in a 
natural way. We leave this issue for the future publication. 
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Figure 11. The dependence of upper bound on 1145 as a function of mAg- Note that U45 drops as ttiab grows to decrease the 
influence of the antisymmetric components of ge towards the up-quark mass relations. 



VII. SUMMARY 



We have investigated flavor constraints and predictions in the presence of the light colored weak singlet scalar, i.e., 
Ag = (3, 1,4/3), that couples antisymmetrically, via Yukawa coupling matrix ge, to the up-quark sector. A tree-level 
exchange of Ag contributes to tt production cross-section in the u-channel and, when below 1 TeV, can thus enhance 
the SM prediction of the forward-backward asymmetry A"^ while not altering the production cross-section CTj^ . This 
relates the strength of gg^ to the Ag mass via \gl'^\ = 0.9 -I- 2.5{mAe/^ TeV). 

Flavor processes that are sensitive to Ag exchange offer a way to place bounds on the remaining two couplings 
Namely, the box diagram consisting of Ag and i-quark exchanges mediates D^-D^ transitions that allow for 



bound extraction of g^^. The bound is phase dependent as summarized in Fig. p^but, \g'^^\ cannot exceed 3 



10- 
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for TTiAg = 1 TeV. The CDF search for resonances in the invariant mass spectrum of di-jets as well as the single top 
production cross-section measurements at the Tevatron put bounds on the coupling g|^ . We show the relevant bounds 
on|gi2|inFig.[6| 

Having obtained the upper bounds on g|^ and g^^ we have also assessed the prospects of looking for radiative top 
quark decays at the LHC. For Ag masses below 1 TeV scale we have found the maximal branching fractions of i — > C7 
and i —> cG of the order of 10^^, whereas t — > U7 and t -^ uG branching fractions are suppressed by an additional 6 
orders of magnitude, owing to strong constraints coming from D^-D^ mixing. 

These bounds offer an opportunity to constrain the up-quark Yukawa sector if the weak singlet color scalar has a 
GUT origin. We have addressed these constraints within a general SU{5) framework that employs one 5- and one 
45-dimensional Higgs representation. Since ge is tied to the antisymmetric contribution of the Higgs doublet in the 45- 
dimensional representation we have deduced the form of the symmetric Yukawa couplings of the 5-dimensional Higgs 
representation to the matter. We have first described generic features of and correlations between the symmetric 
Yukawa couplings that are then confirmed numerically by using a particular — theoretically well-motivated — GUT 
model. We have finally specified the allowed ranges of both symmetric and antisymmetric Yukawa couplings in units 
of mt within that framework. Our setup is unique since the low-energy phenomenological constraints allow us to 
specify Yukawa couplings in the up-quark sector with great accuracy at the scale of unification. 
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